This application claims Paris Convention priority of DE 299 18 341 filed Oct. 18, 1999 the complete disclosure of which is hereby incorporated by reference.
The present invention concerns a device for determining the position of measuring points of a measuring object, comprising a measuring means for determining the position of the measuring points relative to the position of the measuring means, and a reference measuring means for determining the position of the measuring means relative to a reference system.
A device of this kind is disclosed in DE 197 21 903 C1. In the device disclosed therein, the measuring means is formed as a probe with a probe tip. The measuring points of a measuring object are measured by touching the individual measuring points, one after the other, with the probe tip. The position of the probe in a reference system is determined by a reference measuring means rigidly connected to the probe.
The reference measuring means comprises three electronic cameras displaced from one another which detect special measuring symbols in the reference system. The position of the probe tip in the reference system is determined photogrammetrically from the position of the measuring symbols. The conventional device can thereby determine the position of the probe tip and therefore the position of a measuring point relative to the reference system. This can be done, in particular, online, wherein the signal values of the cameras for each measuring point are passed on to a processing means, in which the positions of the measuring points are determined.
If the shape and the position (position and orientation) of at least part of a measuring object is to be determined in three-dimensional space, the conventional device has to detect the position of at least six measuring points of the measuring object, one after the other. If e.g. the shape and position of a circular depression in the measuring object has to be measured, the probe tip must be initially positioned to at least three different measuring points, one after another, from the inside to the edge of the depression for determining the shape of the depression, and the positions of the measuring points must be determined by the processing means. For determining the position of the depress ion in three-dimensional space, the probe tip is subsequently disposed at at least three different measuring points, one after another, outside of the depression on a surface of the measuring object and the positions of the measuring points are again determined. It is easy to imagine that measurement of the shape and position of complex patterns, in particular measuring object geometries which have polygonal substructures, is considerably more demanding.
The recording of a plurality of measuring points for determining the shape and position of an individual measuring object requires a substantial amount of time and work. In addition, the conventional method for determining the position of the measuring points of a measuring object is susceptible to errors generated when the probe tip does not properly seat at a measuring point. This may occur e.g. with poorly accessible measuring points or if determination of the position of the probe tip relative to the reference system is triggered, although the probe tip is still moving towards the measuring point or has already departed therefrom.
A further disadvantage is that the conventional device only allows measurement of measuring objects made from materials which do not yield when the probe tip is supported thereon. For this reason, it is not possible to measure measuring objects made from a flexible material, such as e.g. foamed material or rubber. Nor can patterns on flat surfaces of a measuring object be measured with the conventional device, since a lateral supporting surface for the probe tip is required to determine the shape of the structure (in the above-mentioned example e.g. the edge of the depression) which is not present on a flat surface. For this reason, the patterns disposed on a glass ceramic cooker for delimiting the individual cooking fields cannot be measured with the conventional device.
A further disadvantage of the conventional device is the excessively demanding design of the reference measuring system. At first, a target surface rigidly connected to the reference system must be provided, with the measuring symbols disposed thereon. The rigid connection between the target surface and the reference system must be maintained at all times and the measuring symbols must be disposed very precisely on the target surface, since otherwise the determination of the position of the measuring points would be erroneous. In addition, the photogrammetric method of the conventional device for determining the position of the probe tip from the detected measuring symbols (e.g. so-called inverse photogrammetry) is very demanding and requires an excessive amount of time and calculation.
It is therefore the underlying purpose of the present invention to facilitate, design, and develop a device of the above mentioned kind such that any measuring object, in particular comprising surfaces of any design and consisting of any material, can be measured.
In achieving this object in accordance with the invention and departing from the device of the above mentioned kind:
the reference measuring means is a measuring arm or a robot arm, the proximal end of which is fixed relative to the reference system; and
the measuring means is an optical sensor means disposed at the distal end of the measuring arm or robot arm;
wherein the optical sensor means comprises at least two single or multiple image sensors recording images of at least part of the measuring object; and
the device comprises a processing means which determines the position of the measuring points of the measuring object from the recorded images;
wherein processing of the recorded images is carried out in real time with the recording frequency of the single or multiple image sensors (real time processing).
The device in accordance with the invention advantageously combines a highly accurate and high-speed optical sensor means of simple design with a measuring or robot arm of simple design and with easy handling. A device of simple construction which can measure any measuring object in a straight forward fashion, in particular one having surfaces of any design and consisting of any material, is thereby obtained.
The measuring arm comprises several elements pivotably attached to one another. The proximal end of the measuring arm is disposed in a reference system. The distal end of the measuring arm can be moved with several degrees of freedom. The measuring arm is usually moved manually by a user. The number of positions to which the measuring arm can be moved depends on the number of elements and the number and kind of joints between the elements. The measuring arm comprises sensors which detect a motion of the joints and are preferably disposed in the area of the joints. The sensor signals are guided to a processing means which uses the sensor signals to determine the position of the distal end of the measuring arm relative to the reference system. The measuring arms used for the present invention are sold e.g. by the company Faro, Lake Mary, Fla., USA (http://www.faro.com) under the name FaroArm.
Alternatively, a robot arm can be used instead of a measuring arm, which also comprises several jointed elements and is disposed with its proximal end in a reference system. The distal end of the robot arm can be moved with several degrees of freedom. Towards this end, the robot arm has a number of actuators which are controlled by the processing means such that the distal end of the robot arm assumes a predetermined position. The number of positions to which the robot arm can be moved depends on the number of elements and the number and kind of the joints between the elements. For determining the position of the distal end, the robot arm may have sensors which detect a motion of the joints and which are preferably disposed in the region of the joints. Alternatively, the desired values of the position of the distal end of the robot arm can be used instead of the actual values. The processing means knows the desired values, since it controls the robot arm therewith. Robots with robot arms for use in the present invention are e.g. known from the field of automobile manufacturing in various embodiments and from various manufacturers.
The distal end of the measuring arm or the robot arm has an optical sensor means for determining the position of measuring points of a measuring object relative to the position of the sensor means. The optical sensor means can detect the measuring points of the measuring object in various ways. It can operate with optical radiation of any wavelength. It can detect any particular measuring point individually or several points at the same time. A preferred embodiment of an optical sensor means which is particularly well suited for the inventive device is disclosed in DE 199 10 699.
The optical sensor means comprises at least two single or multiple image sensors. Images of at least part of the measuring object to be measured are recorded using single image sensors (e.g. cameras for recording static images) or multiple image sensors (e.g. cameras for recording dynamic images). The use of at least two single or multiple image sensors permits simple acquisition of measuring object images having larger information content. For recording the images, the single or multiple image sensors must not be disposed at certain positions relative to the measuring points of the measuring object to be measured. The measuring points must merely be contained in the images recorded by the single or multiple image sensors. The images can be recorded with the single or multiple image sensors within a very short time when appropriate hardware and software are used.
The processing means determines the positions of the measuring points of the measuring object from the images recorded by the single or multiple image sensors using suitable image processing methods. Processing of the recorded images keeps pace with the recording frequency of the single or multiple image sensors. Such processing is also referred to as real time processing. Real time processing has the advantage that the image sensors continuously record and evaluate images when the sensor means moves past the measuring object. The recording and processing of images can terminate as soon as the sensor means has measured a certain measuring object, i.e. has received and evaluated images therefrom. In this fashion, a search function can be realized which facilitates automated measurement of measuring objects using the inventive device.
As optical sensor means, single, or multiple image sensors can be used which record an image e.g. every 20 ms or 40 ms, transfer the recorded image to the processing means and store it therein to enable access to the recorded image at any time during further processing. The images recorded in the processing means are then processed in the subsequent processing cycle (20 ms or 40 ms later). The actual recording of the images is carried out more or less simultaneously for all image sensors of the sensor means and takes, in general, less than 2 ms, in particular, approximately 0.5 ms. Through the use of a light flash which emits flashes of light towards the measuring object of approximately 0.02 ms in duration (approximately {fraction (1/100)} of the recording time of the image sensors), the effective recording time can be reduced. Most of the 20 ms or 40 ms is required for reading the recorded images into the processing means.
For every recording of the position of one or more measuring points relative to the position of the optical sensor means by the sensor means, the measuring arm or the robot also determines the position of the sensor means relative to the reference system. From these two position values, the position of the measuring point(s) relative to the reference system can be determined. One can then determine the position of the measuring object or of part of the measuring object relative to the reference system by means of the position of the measuring points.
The device in accordance with the invention can also be operated by untrained staff, since particular measuring points of a measuring object must not be precisely approached to achieve sufficiently accurate measurement of the measuring object. It is sufficient to roughly approach the measuring points of the measuring object and include them in the recorded images. Towards this end, the hardware of the optical sensor means must be adjusted in such a manner that all measuring points of the measuring object to be detected can actually be detected by the sensor means, i.e. the single or multiple image sensors must be directed towards a predetermined measuring volume and that part of the measuring object to be measured has to be disposed in the measuring volume. Selection of the correct measuring points of a recorded measuring object and the determination of the position of the measuring points is then carried out by image processing software in the processing means.
The proposed optical sensor means having high speed processing enables determination of the position of measuring points of a measuring object using one measuring arm. In conventional measuring means, detection of the measuring points requires an excessive amount of time and use in a measuring arm is not practically feasible. A measuring arm guided by a user is always subject to inadvertent shaking by the user which falsifies determination of the position of the measuring points using conventional measuring means. The proposed optical sensor means requires such a short time (e.g. 2 ms, or 0.02 ms using a light flash) for recording images of the measuring object that inadvertent shaking by the user has no effect on the position determination.
After transport of a robot arm, residual vibrations are present at the distal end of the robot arm when the motion has ended. In conventional measuring means, these vibrations cause measuring errors in determination of the positions of the measuring points such that the determination of the position cannot be carried out before the vibrations have stopped. Through the use of the proposed optical sensor means, determination of the position can be advantageously started during these residual vibrations, since the optical sensor means requires such a short time for recording the images of the measuring object that vibrations have no effect on the determination of the position.
An advantageous further development of the present invention proposes that the processing means determines the position of the measuring points of the measuring object for each of the images recorded by the single or multiple image sensors, wherein the processing means comprises first means for filtering out the best positions of the measuring points. Filtering out of the best positions can be effected e.g. by averaging the positions determined on a measuring object and by eliminating those positions which excessively differ from the average value. Filtering out of the best positions can also be carried out using the standard deviation of the positions of the measuring points determined on a measuring object. In this manner, anomalous values can be filtered out of the measurements. The effects of inadvertent shaking of the measuring arm by a user or the effects of vibrations of the robot arm on the determination of the positions of the measuring points can be further reduced using the inventive device.
In a further advantageous embodiment of the present invention, the processing device comprises second means for determining the shape and also the position of at least one part of the measuring object relative to the position of the optical sensor means using the positions of the measuring points. By way of example, that part of a measuring object whose shape and position (position and orientation) is to be determined could e.g. be the colored borders of the cooking fields on a glass ceramic cooker, the proper positioning of which on the glass ceramic cooker is to be checked. A further typical application of the inventive device is measurement of the tread of a vehicle tire. A particularly preferred field of application is the vehicle industry, where the position (position and orientation) of a leaf member (car door, trunk or hood) relative to the vehicle chassis can be checked using the inventive device.
The second means of the processing means are advantageously designed such that they substantially simultaneously determine both the shape and the position of at least one part of the measuring object relative to the position of the optical sensor means. This has the advantage that a plurality of sequential separate measurements are not required for detecting the shape and the position. Instead, the corresponding measuring points for the measurements are detected substantially at the same time. Within the content of the present invention, xe2x80x9csubstantially at the same timexe2x80x9d means that measurements of the shape and position of the measuring object are carried out rapidly, one after the other such that the measurements can be associated with the same value for the position of the sensor means relative to the reference system without loss of accuracy.
The single or multiple image sensors are preferably designed as electronic cameras, in particular CCD cameras. CCD cameras contain so-called CCD image converters comprising a plurality of semi-conductor components, disposed in the shape of a matrix, and functioning as photo-electric light receivers. The light receivers decompose the optical images projected by a camera lens, into single image elements with subsequent conversion into electric signals. The output signals of CCD cameras are already present in digital form and must not be digitized for further processing in the processing means.
In accordance with a preferred embodiment of the present invention, the single or multiple image sensors each record at least two images with two-dimensional information content of at least part of the measuring object and the processing means combines the two-dimensional images into at least one image having three-dimensional information content and evaluates the image to determine the shape and the position of the recorded part of the measuring object relative to the position of the optical sensor means. Each recorded two-dimensional image has associated values, recorded by the measuring arm or the robot arm, for the position of the sensor means relative to the reference system. The two two-dimensional images representing the same part of the measuring object are combined into an image having three-dimensional information content. This image must not be present in the form of a three-dimensional image on which the recorded measuring object is shown in a three-dimensional manner. Rather, the three-dimensional image can also simply be a mathematic model by means of which the shape and position of the recorded part of the measuring object is determined by the processing means.
In accordance with a preferred embodiment of the present invention, after being activated, the device continues to record images with the optical sensor means at the recording frequency and processes same in the processing means, wherein the processing means comprises third means for classifying the determined positions of the measuring points of the measuring object as valid or invalid depending on whether the determined positions are within or outside of a predetermined measuring volume of the optical sensor means.
In the inventive device in accordance with this embodiment, the optical sensor means is manually moved by a user on a measuring arm or moved on a robot arm by a robot towards the measuring object to be measured. After activation of the device, the single or multiple image sensors record images at a scanning frequency which are then processed by the processing means in real time. As long as the positions of the measuring points of the measuring object are outside of a predetermined measuring volume, the positions are classified as invalid and new images are recorded. As soon as the processing means detects that the positions of the measuring points are within the measuring volume, the positions are classified as valid. Determination of the positions can then be terminated or continued until a predetermined number of valid measurements of the same measuring object are obtained.
The device advantageously comprises signalling means for external communication of valid determination of positions of the measuring points of a measuring object. If a measuring arm is used as reference measuring means, the signalling means may be designed e.g. as display means (signal lamp or symbol on a screen) or acoustic means (signal tone) to inform the user of a valid position determination. If a robot arm is used as the reference measuring means, the signalling means can inform the robot in an appropriate fashion of a valid determination of measuring points. The user or the robot can then terminate the current position determination and e.g. move the optical sensor means into a rest position and deactivate the inventive device.
Alternatively, after completion of the position determination of the measuring points of a measuring object, the optical sensor means can be moved to a further measuring object where renewed position determination of measuring points of the further measuring object is carried out. Towards this end, an advantageous further development of the inventive device proposes that the processing means comprises a processing control in which several objects to be measured are determined in a defined sequence, wherein, after termination of position determination of the measuring points of a measuring object to be measured, processing control automatically changes to the next measuring object. Upon completion of position determination of the measuring points of the last measuring object to be measured, as defined by the processing control, processing control terminates the position determination and the inventive device is deactivated.
The single or multiple image sensors of the optical sensor means work in a predetermined measuring volume. The single or multiple image sensors have a defined resolution only within this measuring volume. The position of the measuring object can only be determined with the required accuracy when the measuring points of the measuring object are positioned within the measuring volume. Towards this end, the sensor means is positioned by the user or the robot at such a separation from the part of the measuring object to be measured that the measuring points to be recorded are located in the measuring volume.
A screen is connected to the optical sensor means for monitoring the measuring volume in two dimensions. The screen shows the output signals of the single or multiple image sensors, e.g. the field of view detected by the single or multiple image sensors. The optical sensor means at the distal end of the measuring arm or the robot arm is displaced until all measuring points of the measuring object to be recorded are visible on the screen. This facilitates positioning of the sensor means by the user or by the robot such that the measuring points of the measuring object lie, in two dimensions, within the measuring volume shown on the screen.
For monitoring the measuring volume in the third dimension perpendicular to the two dimensions shown on the screen, an advantageous further embodiment of the invention proposes that the optical sensor means comprises at least one laser for determining the separation between the optical sensor device and a surface of the measuring object using a laser beam. If the separation is also set to a defined value in the third dimension, a location of the measuring points within the measuring volume of the optical sensor means is ensured.
The separation between the sensor means and the surface of the measuring object can be measured in different ways. The invention suggests, in particular, that the optical sensor means comprises a laser for emitting a laser beam onto the surface of the measuring object, an optical sensor for receiving the laser beam reflected from the surface of the measuring object, and means for measuring the travel time of the laser beam from the laser to the optical sensor and for determining the separation between the optical sensor means and the surface of the measuring object from the travel time of the laser beam. The optical sensor is e.g. an area sensor. The detected separation between the sensor means and the surface of the measuring object is indicated to the user e.g. on the screen or on another display device or is transmitted to the processing means for controlling the robot. The user or the robot displaces the sensor means until it is located at a predetermined separation from the surface of the measuring object i.e. within the measuring volume.
Alternatively, the optical sensor means can have two lasers for emitting two laser beams towards the surface of the measuring object, wherein the laser beams extend in a common plane and cross one another at a common point of intersection disposed within the measuring volume of the optical sensor means. The separation between the images of the laser beams varies in dependence on the separation between the sensor means and the surface of the measuring object. When the sensor means is disposed at the predetermined separation from the surface, the two images of the laser beams coincide and only one single image is shown on the surface of the measuring object. The user can follow the laser beam images on the surface of the measuring object. The user can displace the sensor means until only one laser beam image is visible.
Monitoring of the separation between the two laser beam images can also be carried out automatically using an area sensor and not by the user. A preferred embodiment of the invention thereby proposes that the optical sensor means comprise an area sensor for recording the images of the two laser beams on the surface of the measuring object and means for measuring the separation between the images of the laser beams and for determining the separation between the optical sensor means and the surface of the measuring object from the separation of the images of the laser beams. Towards this end, the trigonometric relationships between the lasers and the area sensor must be known. The detected separation between the optical sensor means and the surface of the measuring object can be indicated to the user e.g. on a screen or other display means. Alternatively, the detected separation can be transmitted to the processing means which correspondingly displaces the sensor means at the distal end of the robot arm.
Using the above described devices, a user or a robot can easily move the optical sensor means to a position relative to the measuring points of the measuring object such that the measuring points are located in the measuring volume of the optical sensor means. This ensures exact position determination of the measuring points of the measuring object relative to the reference system.
The at least one area sensor can be at least one single or multiple image sensor, e.g. a CCD camera, for receiving the laser beam reflected from the surface of the measuring object or for receiving the images of the laser beams on the surface of the measuring object.
A preferred embodiment of the present invention is described below with reference to the drawings.